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AMENDMENT 



Assistant Commissioner for Patents 
Washington, D.C. 20231 

Dear Sir: 

In response to the Office Action dated March 9, 2000 (Paper No. 12) in the above- 
referenced patent application, please make the following amendments: 

IN THE CLAIMS: 

Please amend Claims 1, 3, 4, 8, 11, 17, 18, 20 and 23: 

1. (Thrice Amended) A computerized method of [extracting] identifying a key frame from 
a video, comprising: 

a) providing a reference frame; 

b) providing a current frame different from the reference frame; 

c) determiiiing a chromatic difference measure between the reference frame and the 
current frame; 

d) determining a structure difference measure between the reference frame and the 
current frame based, at least in part, on edges identified in each of the frames; [and] 
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e) identifying the current frame as a key frame if the chromatic difference measure 
exceeds a chromatic threshold and the structure difference measure exceeds a structure 
threshol d, otherwise selecting a new current frame; and 

f) repeating c). d), and e) until a key frame is identified . 

3 . (Twice amended) The method defined in Claim [ 1 ] 2, additionally comprising 
repeating [c)-e)] bVf) for a new current frame until another key frame is identified or the end of the 
video is reached. 

4. (Amended) The method defined in Claim [3] 1, wherein the new current frame is 
selected to be at a predetermined time interval after the current frame. 

8. (Thrice Amended) A computerized method of [extracting] identifying a key frame from 
a video having a plurality of frames, the method comprising: 

a) providing a reference frame; 

b) providing a current frame different from the reference frame; 

c) determining a first difference measure between the reference frame and the 
current frame; 

d) determining a second difference measure between the reference frame and the 
current frame based, at least in part, on edges identified in each of the frames; [and] 

e) identifying the current frame as a key frame if the first difference measure 
exceeds a first threshold and the second difference measure exceeds a second threshold,, 
otherwise selecting a new current frame; and 

f) repeating c\ d), and e) until a key frame is identified . 

11. (Twice Amended) The method defined in Claim [8] 9, additionally comprising 
repeating [c)-e)] b)-f) for a new current frame until another key frame is identified or the end of the 
video is reached. 



17. (Twice Amended) The method defined in Claim 8, additionally comprising 
determining a third difference measure between the reference frame and the current frame, and 
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wherein the identifying identifies the current frame as the key frame if the third difference measure 
exceeds a third threshol d, otherwise selecting a new current frame . 

18. (Thrice Amended) A computerized method of [extracting] identifying a key frame from 
a video having a plurality of frames, the method comprising: 

a) providing a reference frame; 

b) providing a current frame different from the reference frame; 

c) determining a structure difference measure between the reference frame and the 
current frame based, at least in part, on edges identified in each of the frames; [and] 

d) identifying the current frame as a key frame if the structure difference measure 
exceeds a structure threshol d, otherwise selecting a new current frame; 

e) repeating c) and d) until a key frame is identified . 

20. (Twice Amended) The method defined in Claim [18] 19, additionally comprising 
repeating [c) and d)] bVe) for a new current frame until another key frame is identified or the end of 
the video is reached. 

23 . (Amended) A computerized method of [extracting] identifying a key frame from a video 
having a sequence of frames, the method comprising: 

a) providing a reference frame; 

b) providing a current frame different from the reference frame; 

c) determining a chromatic difference measure between the reference frame and the 
current frame; 

d) determining a structure difference measure between the reference frame and the 
current frame; [and] 

e) identifying the current frame as a key frame if the chromatic difference measure 
exceeds a chromatic threshold and the structure difference measure exceeds a structure 
threshold, [without accumulating differences between pairs of frames of the video 
sequence] otherwise selecting a new current frame: and 

f) repeating c). d). and e) until a key frame is identified . 
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Please add new Claim 24 as follows: 

24. A computerized method of identifying a key frame from a video, comprising: 

a) providing a reference frame; 

b) providing a current frame different from the reference frame; 

c) determining a chromatic difference measure between the reference frame and the 
current frame; 

d) determining if the chromatic difference measure exceeds a chromatic threshold; 

e) identifying the current frame as a key frame candidate, otherwise selecting a 
new current frame and skipping f) and g); 

f) deteirnining a structure difference measure between the reference frame and the 
key frame candidate based, at least in part, on edges identified in each of the frames; 

g) identifying the key frame candidate as a key frame if the structure difference 
measure exceeds a structure threshold, otherwise selecting a new current frame; and 

h) repeating c) through g) until a key frame is identified. 

REMARKS 

Applicant adds new Claim 24 and amends Claims 1, 3, 4, 8, 1 1, 17, 18, 20 and 23 by this 
paper. Claims 2, 5-7, 9-10, 12-16, 19, and 21-22 remain unchanged and are also presented for 
examination. Reconsideration and allowance of all Claims 1-24 in light of the present remarks is 
respectfully requested. 

Discussion of the Question Regarding a Journal Article 

The Examiner asked why the article "Production Model Based Digital Video 
Segmentation" as it appears in the Multimedia Tools and Applications, 1995, by Aran Hampapur, 
Ramesh Jain and Terry Weymouth, as referenced in Zabih et al., U.S. Patent No. 5,767,922, was 
not disclosed in an Information Disclosure Statement. The article seems to be cumulative to 
Chapter 5 of the Hampapur Dissertation, University of Michigan, 1995, of which Jain and 
Weymouth were the co-chairs of the doctoral committee. However, Applicant's representative will 
be submitting the article in a Supplemental Information Disclosure Statement. 
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Discussion of the Claim Rejection under 35 U.S.C. § 102(e) 

Claims 18-22 were rejected under 35 U.S.C. § 102(e) as being anticipated by Zabih et al. 
("Zabih"), U.S. Patent No. 5,767,922. Zabih describes "detecting scene breaks in a sequence of 
video frames providing a moving picture. Entering and exiting edge pixels in each of a plurality 
of successive video frames are counted, and an edge change fraction for each of the successive 
video frames is derived therefrom. Peaks which are detected in the edge change fractions are 
indicative of scene breaks." The scene breaks include cuts, fades, dissolves and wipes. 

Applicant's claimed invention is directed to a computerized method of extracting a key 
frame from a video having a plurality of frames. In contrast, Zabih identifies scene breaks which 
are typically five to ten or more frames in length. See, for example, the scene break peaks 
identified in Figure 7, each of which include multiple frames. Zabih does not describe how the 
system identifies a key frame - is it one of the frames from the scene break, is it the middle frame 
from a shot identified by a pair of successive scene breaks, is more than one frame selected from a 
shot, etc. 

Applicant's claimed invention performs the key frame extraction by comparing a current 
frame to a reference frame. Some of the claims have been amended to clarify that a new current 
frame is selected if the structure difference measure does not exceed the structure threshold. Acts 
c) and d) are repeated until a key frame is identified. The reference frame stays fixed until the 
key frame is identified (Claim 19). In contrast, Zabih performs calculations on successive 
frames, as stated at Column 4, lines 49-52: "In order to detect scene breaks efficiently and 
accurately, in accordance with the present invention an edge change fraction is obtained in each 
of a plurality of successive video frames. . .". Applicant's claimed invention does not calculate 
an edge change fraction between successive frames, as performed by Zabih. 

Since Zabih does not describe a computerized method of extracting a key frame from a 
video where the reference frame is fixed until a key frame is identified, Applicant submits that 
Zabih is overcome as a reference for Claim 18. Since Claims 19-22 are dependent on 
independent Claim 18, pursuant to 35 U.S.C. § 1 12, |4, they incorporate by reference all the 
limitations of the claim to which they refer. Therefore, the rejection of the dependent Claims 19- 
22 has also been overcome. Therefore, in view of the above, it is submitted that Claims 18-22 are 
clearly distinguished from the cited art and are patentable. 
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Discussion of the Claim Rejection under 35 U.S.C. § 103(a) 

Claims 1-17 and 23 were rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Zabih, U.S. Patent No. 5,767,922, in view of the submitted Dissertation by Arun Hampapur, 
University of Michigan, 1995 ("Hampapur"). The Examiner cited pages 97-102 of the dissertation, 
and specifically page 97, lines 11-18. 

Applicant's claimed invention performs the key frame extraction by comparing a current 
frame to a reference frame. Some of the claims have been amended to clarify that a new current 
frame is selected if the chromatic difference measure does not exceed the chromatic threshold and 
the structure difference measure does not exceed the structure threshold. Acts c) through e) are 
repeated until a key frame is identified. The reference frame stays fixed until the key frame is 
identified (Claims 2, 9). In contrast, Zabih performs calculations on successive frames, as stated 
at Column 4, lines 49-52. Applicant's claimed invention does not calculate an edge change 
fraction between successive frames, as performed by Zabih. 

The Examiner stated that it would be obvious "to incorporate the Hampapur chromatic 
scaling teaching of the dissertation into the Zabih method in order to have dual measures, 
because no method works perfectly by itself (Zabih: column 16, lines 30-45)." The cited text is 
as follows: 

Our edge detection method does not handle rapid changes in overall scene 
brightness, or scenes which are very dark or very bright. Rapid changes in overall scene 
brightness can cause a false positive. Since a thresholded gradient-based edge detector is 
dependent on the relative contrast of regions in the image, large-scale scalings in image 
brightness will disturb the edge density of the scene. This effect sometimes occurs in 
scenes due to camera auto gain. 

Scene break detectors of the prior art based on intensity histogramming will 
also generate false positives when the overall scene brightness changes 
dramatically. Although the intensities change dramatically, the underlying edge 
structure of the image does not change. A more robust edge detection scheme may 
enable these events to be handled. 

Applicant respectfully disagrees and submits that the cited passage teaches away from use 
of an intensity histogram and, instead, promotes a more robust edge detection scheme. Zabih does 
not describe how dual measures, dual thresholds and the sequence of applying the dual measures 
and the dual thresholds would be implemented. 

Starting at column 12, line 60 through column 16, the Zabih reference tests its method of 
determining edge change fractions on sample video sequences against those of other methods (e.g., 
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Nagasaka, Otsuji, Zhang, Hampapur). See the three sets of test results in conjunction with Figures 
14-16, Figures 17-18, and Figures 19-20. Column 14, lines 34-38 describe an instance of a 
shortcoming of the Hampapur method, for example. The description of the contrasted test results is 
also used by the Examiner to reject dependent Claims 7 and 14-17. Regarding Claims 7 and 14, the 
Examiner stated that "the Zabih method, now incorporating the Hampapur chromatic scaling 
method as outlined in the dissertation, discloses that the second difference measure is only 
performed if the first difference measure exceeds the first threshold (Zabih: column 15, lines 1- 
20)". Applicant respectfully disagrees and submits that the cited text merely describes test results 
for two separate methods. Regarding Claims 15-16, the Examiner cited column 15, lines 30-68. 
Applicant respectfully disagrees and submits that the cited text describes test results for Zabih' s 
method of determining edge change fractions without motion compensation and then with motion 
compensation enabled. The motion compensation is not a difference measure, but is done prior to 
computing the edge change fractions to digitally align successive frames. Regarding Claim 17, the 
text cited by the Examiner further describes motion compensation, which again is not a difference 
measure. 

Note that the rejection of Claim 10 on page 6 of the Office Action dated March 9, 2000, 
appears to be in error. Claim 10 recites that the "first difference measure is orthogonal to the 
second difference measure", and is not similar to Claim 3. 

Since Zabih does not describe a computerized method of extracting a key frame from a 
video where the reference frame is fixed until a key frame is identified, Applicant submits that 
Zabih is overcome as a reference for Claims 1, 8 and 23. Since Claims 2-7 and 9-17 are 
dependent on independent Claims 1 and 8, respectively, pursuant to 35 U.S.C. § 112, f4, they 
incorporate by reference all the limitations of the claim to which they refer. Therefore, the 
rejection of the dependent Claims 2-7 and 9-17 has also been overcome. Therefore, in view of 
the above, it is submitted that Claims 1-17 and 23 are clearly distinguished from the cited art and 
are patentable. 

New Claim 

New Claim 24 is supported by Figure 5 and the corresponding text at pages 9-11 of the 
patent specification. 
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Conclusion 

By this amendment, Applicant has added a new claim and amended the claims. In view of 
the foregoing amendments and remarks, Applicant respectfully submits that Claims 1-24 of the 
above-identified application are in condition for allowance. However, if the Examiner finds any 
further impediment to allowing all claims that can be resolved by telephone, the Examiner is 
respectfully requested to call the undersigned. 

Respectfully submitted, 

KNOBBE, MARTENS, OLSON & BEAR, LLP 
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/ ' Raimond J. Salenieks 

Registration No. 37,924 
Agent of Record 
620 Newport Center Drive 
Sixteenth Floor 
Newport Beach, CA 92660 
(619) 235-8550 
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